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Abstract of EP0427358 

An interactive system for guiding the use of a 
surgical tool uses at least one innaging technique, 
such as CT scanning. A mechanical arm has a 
fixed base at a first end and a tool holder that 
holds the surgical tool at a second end. A display 
displays one or more images from the image 
space of a patient's anatomy. A computer is 
coupled to the display and the mechanical arm. 
The computer tracks the location of the surgical 
tool through physical space, performs a 
transforming rotation of the physical space to the 
image space, and causes the display to display 
the location of the surgical tool within the image 
space. 
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0 Interactive image-guided surgical system. 

0 An Interactive system for guiding the use of a 
surgical tool uses at least one innaging teciinique, 
such as CT scanning. A mechanical ami has a fixed 
base at a first end and a tool holder that holds the 
surgical tool at a second end. A display displays one 
or more images from tiie image space of a patient's 
anatomy. A computer is coupled to the display and 
the mechanical ami. The computer tracks the loca- 
tion of the surgical tool through physical space, 
performs a transfomning rotation of the physical 
space to the image space, and causes the display to 
display tiie location of the surgical tool witiiin the 
Image space. 
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INTERACTIVE IMAGE-GUIDED SURGICAL SYSTEM 



This application is a continuation-in-part of U.S. 
Application Serial No. 119,353, filed on Novenriber 
10, 1987. 



FIELD OF THE INVENTION 

The present invention relates to a system for 
guiding a surgeon interactively through a surgical 
procedure, and more particularly to an integrated 
system of hardware and software which allows for 
the intuitive use of imaging-derived data regarding 
a patient's anatomy in order to guide a surgeon 
through a surgical procedure. 



BACKGROUND OF THE INVENTION 

There are a number of well-known diagnostic 
Imaging techniques that allow a physician to obtain 
high fidelity views of the human body. Imaging 
systems which provide cross-sectional 
(tomographic) views of anatomical structure without 
invasive procedures include computed tomography 
(CT) x-ray imagers and magnetic resonance (MR) 
imagers. 

A problem associated with the scanning tech- 
niques is that each imaging process is sensitive to 
the patient's position within the imaging device. 
Therefore, each set of images has a discrete, 
unique orientation. Thus, images formed from the 
same modality at different times and Images 
fomned at essentially the same time, but from dif- 
ferent imaging modalities (for example, CT and 
MRI) cannot be compared on a point-by-point ba- 
sis. This prevents accurate comparison of regions 
within the images. 

A surgeon also deals with orientation differ- 
ences to the imaging space. For example, although 
a neurosurgeon will know where his surgical tool is 
with regards to certain anatomic landmarks he may 
not know with the desired precision where the tool 
is with regards to the lesion visible on the Images. 
There have been attempts to solve this problem by 
temporary attachment of a relatively large brace- 
like structure surgically attached to portions of the 
body, such as the head. By orienting a surgical tool 
with respect to this structure, and by knowing the 
location of internal anatomical areas of interest with 
relation to this attached structure, the pcsition of 
the surgical tool with respect to the anatomical 
areas of interest will be known. 

A problem with these structures is their size 
and their interference with normal daily activities, 
such as sleeping. The structures are therefore not 



used over a long period of time (e.g. for more than 
12 hours) so that a comparison of Images, or the 
location of a specific point within the anatomy, 
taken over a substantial time period is not practical. 
5 There is therefore a need for an interactive 
system which will guide a surgeon in the manipula- 
tion of a surgical tool to an exact location that is 
specified by an imaging system. 

10 

SUMMARY OF THE INVENTION 

This and other needs are satisfied by the 
present invention which provides an arrangement 

IS for an interactive image-guided surgical system. 
The system according to the present invention 
defines an intemal coordinate system within the 
anatomy of a patient. The internal coordinate sys- 
tem is located with respect to an external coordi- 

20 nate system, for example, by locating the end tip of 
a surgical tool with a known reference point in the 
internal coordinate system. Once the position of the 
intemal coordinate system is known with respect to 
the external coordinate system, the surgical tool 

25 can be moved anywhere within either the external 
or intemal coordinate systems and Its location will 
be known with a high amount of precision. 

The present invention also relates to an arm 
that a surgical tool is attached to for use in an 

30 image-guided system. The arm carries position en- 
coders so that as the arm is moved, the location of 
the surgical tool that is attached to the arm will 
always be known with respect to the internal co- 
ordinate system. 

35 

BRIEF DESCRIPTION OF THE DRAWINGS 

RGS. 1A and IB illustrate different views of a 
40 head having fiducial implants located In the 
head. 

FIG. 2 shows an operating arrangement In ac- 
cordance with an embodiment of the present 
invention, 

45 FIG. 3 shows a mechanical arm constructed in 
accordance with an embodiment of the present 
invention. 

FIG. 4 shows an enlarged view of gear engage- 
ment for an optical encoder used with the arm 
50 of FIG. 3. 

FIG. 5 is a flow chart for an interactive, image- 
guided surgical system In accordance with an 
embodiment of the present invention. 
FIG. 6 Is a flow chart of the pre-operative soft- 
ware. 



2 



3 



EP 0 427 358 A1 



4 



FIG. 7 is a flow chart of the first function of the 
preoperative software. 

FIG. 8 is a flow chart of the second function of 
the preoperative software. 
FIG. 9 Is a flow chart of a third function of the 
preoperative software. 

FIG. 10 is a flow chart of the fourth and fifth 

functions of the pre-operative software. 

FIG. 1 1 Is a flow chart of the sixth function of 

the preoperative software. 

FIG. 12 is a flow chart showing the selection of 

functions in the intra-operative software. 

FIG. 13 is a flow chart of the first function in the 

intra-operative software. 

FIG. 14 is a flow chart of the second function of 

the intra-operative software. 

FIG. 15 is a flow chart of the fourth function of 

the intra-operative software. 

FIG. 16 is a flow chart of the fifth function of the. 

intraoperative software, 

DETAILED DESCRIPTION 

The present invention defines a three-dimen- 
sional internal coordinate system that is fixed within 
a human anatomy. The internal coordinate system 
is established within the anatomy using the present 
Invention by affixing three or more fldudal Implants 
to portions of the anatomy. The fiducial implants 
are affixed to points which will not change their 
spatial relationship to one another over a relatively 
long period of time, such as a few months. 

An example of placement of fiducial implants in 
the anatomy are shown in FIGS. 1A and IB. In 
these drawings, fiducial Implants 10A,10B,10C are 
implanted in three separate, spaced locations with- 
in a skull 18. 

Since these three fiducial implants 10A-C are 
arranged in a noncollinear manner, a plane Is 
forced which contains these fiducial implants 10A- 
C. Once a plane is defined, a three-dimensional 
coordinate system is defined. Any point within the 
body will be within the internal coordinate system. 

Although fiducial implants are shown, any three 
points that are affixed with respect to the region of 
Interest can comprise the three points used to 
define the internal coordinate system. However, 
fiducial implants 10A-G that are identifiable and 
measurable by different imaging systems, such as 
CT imagers and MRI imagers, are preferred. The 
fiducial implants 10A-C can be relatively small and 
unobtrusive so that no discomfort or self-con- 
sciousness will be experienced by the patient even 
though the patient may canry the Implants 10A-C 
for a relatively long period of time. 

A scan, using a known imaging technique, Is 
perfomied once a patient has the three fiducial 
implants 10A-C implanted. An internal coordinate 



system will then be defined with respect to these 
three fiducial implants 10A-G, During subsequent 
scans, whether after a few minutes or a few 
months, the patient's orientation may change rela- 

5 tive to the imaging apparatus. However, using the 
present invention, this new orientation can be mea- 
sured by locating the fiducial Implants 10A-C in 
relation to the imaging apparatus and comparing 
their locations to the previously recorded locations. 

70 This comparison technique permits re-orientIng Im- 
ages of subsequent scans to a position corre- 
sponding to the earlier recorded scan so that 
equivalent image slices can be compared. 

Regardless of the reason why the patient is 

T5 oriented differently, by taking advantage of the 
fixed, fully-defined internal coordinate system in 
the anatomy, the location and direction of the plane 
defined by the three fiducial implants 10A-C in the 
first imaging session can be compared with the 

20 location and direction of the same plane defined by 
the three fiducial implants at the time of the second 
Imaging session. The cartesian systems are 
aligned by three independent rotations. The trans- 
lation of one cartesian coordinate system to an- 

25 other is a well-known technique and readily per- 
formable by modern computers. An example of an 
arrangement which defines an internal coordinate 
system for the anatomy and perfonns a transforma- 
tion with respect to rotation from one cartesian 

30 coordinate system to another is described in U.S. 
Patent Application 119,353 filed on November 10, 
1987 for a "Method and Apparatus for Imaging the 
Anatomy", and is herein expressly incorporated by 
reference. 

35 Once the internal coordinate system is estab- 
lished, an external coordinate system is also there- 
by established by the three noncollinear fiducial 
Implants 10A-C. In order to keep track of a moving 
point In both the internal and the external coordi- 

40 nate systems, It is merely necessary for the sys- 
tem to initially establish the location of that point 
with respect to a point In the Internal coordinate 
system and then continuously follow the movement 
of the point in the external coordinate system. As 

45 an example, assume that tiie point In the external 
coordinate system is the end tip of a laser. In order 
to keep track of tiie location of that end tip in both 
tiie external and internal coordinate systems, the 
end tip of the laser is first brought into a known 

50 relationship with one of the fiducial implants 10A-C. 
for example touching the implant, and the com- 
puter notes tills initialization. The computer used 
with tiie imaging system then follows the location 
of the end tip as the laser is moved anywhere 

55 wltiiin the Internal and external coordinate systems. 
A positioning encoder tracks the position of the end 
tip and feeds signals relating the movement of the 
end tip within the coordinate systems to the com- 
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puter. Since the original position of the end tip (\.e. 
against the fiducial Implant lOA-C) is known, and 
its movements have been continuously tracked and 
fed to the computer since the original position of 
the end tip was entered Into the computer, the 
position of the end tip in either the internal or 
external coordinate systems will be known at all 
times. 

FIG, 2 shows a schematic illustration of an 
operating environment according to an embodiment 
of the present invention. In this figure, a patient has 
fiducial implants 10 A-C implanted in the skull 18. 
An imager 102 operates as described earlier in 
conjunction with a programmable computer 104. An 
operator control panel 110 is coupled to the prog- 
rammable computer 104, as is a display 108 which 
includes a target display 112 that displays the 
coordinates of a target (used In radiation therapy 
applications). 

An external arm 34 is fixed to a base 36. The 
arm 34 cam'es a tool 38 which is changeable, and 
can be for example, a laser or any of a number of 
surgical tools, such as a pointer, ultrasound unit, 
biopsy probe, radiation beam collimator, etc. The 
arm 34 has a number of joints 42, although only 
one is shown for purposes of illustration in FIG. 2. 
The movement of the arm 34 is tracked by com- 
puter 104 so that the position of the tool 38 relative 
to the base 36 of the arm 34 will always be known. 
The movement of the tool 38 through the external 
and internal coordinate systems (with reference to 
the base 36 of the external coordinate system) will 
be known precisely using the following method. 

At the end tip 39 of the tool 38 a sensor 40 
may be located. The sensor 40 can be a metal 
detector or an ultrasonic detector, or any instru- 
ment that can sense the position of a fiducial 
implant 10 A-C in the patient. If the fiducial im- 
plants 10 A-C are placed in the skull 18. the sensor 
40 at the end tip 39 of the tool 38 is moved by the 
arm 34, under the guidance of the surgeon, until it 
contacts a fiducial implant 10 in the skull 18. This 
contact of the end tip 39 with the fiducial implant 
10 is noted by the computer so that the initial 
position of the end tip 39 relative to the internal 
coordinate system is known. Furthermore, since 
the position of the end tip 39 relative to the base 
36 in the extemal coordinate system is also always 
tracked and known, the position of the end tip 39 
can be followed through both extemal and internal 
coordinate systems following the initialization of 
placing the end tip 39 Into contact with the fiducial 
implant 10. 

The means to track the arm 34 is well known 
and is accomplished by sensors (not shown in FIG. 
2) in various locations of the arm 34. detecting 
either rotation or movement of the joints 42 of the 
arm 34. 



In surgery, the internal coordinate system de- 
fined by the three fiducial implants 10A-C allow, for 
example, a laser to be followed as it cuts through 
tissue to a tumor. The imaging system 102 used in 

5 the imaging procedure is positioned to continually 
take imaging data that is provided to the computer 
104 and the display 108 to guide the surgeon who 
manipulates the arm 34 and the laser used as the 
surgical tool 38. As the laser cuts through the 

10 tissue, the change in the tissue will be apparent in 
the display 108 of imaging system and can be 
followed with respect to the fixed internal coordi- 
nate system. 

An example of a mechanical arm whose move- 

75 ments can be tracked and which can hold a variety 
of surgical tools 38 is shown in FIG, 3. The base 36 
of the arm 34 is movably fixed to some location. 
The arm 34 has two arm links 40A.40B. The first 
anT» link 40A is coupled to the base by two gimbal 

20 joints 42. The first arm link 40A therefore has two 
degrees of motion, as provided by the two gimbal 
joints 42. 

A second ami link 40B is coupled to the first 
arm link 40 A by a second pair of gimbal joints 42. 
25 This second pair of gimbal joints 42 provides the 
second arm link 40B witii two additional degrees of 
motion. Relative to the base 36 of the arm 34, the 
second arm link 40B therefore has four degrees of 
motion. 

30 A tool holder 44 is coupled to the second arm 
link 40B tiirough a pair of gimbal joints 42. The tool 
holder 44 can hold any of a number of different 
tools, including a pointer, an ultrasound unit, a 
surgical laser, a biopsy probe, a radiation beam 

35 collimator, etc. The third pair of gimbal joints 42 
provides the tool 38 with two additional degrees of 
motion, so tiiat relative to the base 36, the tool 38 
has six degrees of motion. 

The exact positioning of the tool 38 relative to 

40 the base 36 is kept ti'ack of by optical encoders 46. 
One optical encoder 46 is assigned to each gimbal 
joint 42. As an individual gimbal joint 42 is rotated 
around its pivot, tiie optical encoder 46 determines 
the precise amount of rotation of the gimbal Joint 

45 42 around its pivot. The information from each of 
the six optical encoders 46 is provided to the 
programmable computer 104, which can therefore 
precisely track tiie movement of die tool 38 relative 
to the base 36 by keeping track of the individual 

50 rotations of tiie gimbal joints 42 around their pivots. 
As can be seen in tfie embodiment of FIG. 3, 
the opti'cal encoders 46 are of a size such that they 
can be an^ged within tiie gimbal joint 42. This 
makes for a very compact arm structure and ac- 

65 curate encoding of tiie position of the gimbal joint 
42. The entire arm structure 34 is steriiizable and 
can be made out of stainless steel, for example. 
Furthermore, in order to make the arm 34 easy to 
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manipulate and use, the arm 34 is counterbalanced 
in a conventional manner. 

Although other means of measuring and feed- 
ing back information as to the pivoting or tilting of 
the gimbal joints 42 can be used, an optical en- 
coder such as commercially available and pro- 
duced by Heidenhain or by ITEK are suitable. As 
mentioned before, it is advantageous that the op- 
tical encoders 46 are of a size that fits within the 
gimbal joints. 

A detail of the mounting of an optical encoder 
46 is shown in FIG. 4. A gear 50 that is coupled to 
the gimbal joint 42 meshes at an angle of approxi- 
mate 6. to the gear 52 that drives the optical 
encoder 46. This angled meshing prevents back- 
lash of the gears so that the accuracy of the 
readout of the optical encoder is ensured. 

During an operation, three separate raster im- 
ages and a graphic image are displayed on the 
video display 108 simultaneously to assure the 
surgeon of accurate spatial orientation. Each dif- 
ferent raster image can be supplied by a different 
type of imaging technology. For example, the three 
different raster images supplied simultaneously on 
the video display screen 108 can be from three 
different imaging modalities such as CT, MRI, etc. 
Alternatively, multiple slices from a single imaging 
modality can be displayed simultaneously Instead 
of the same slice from different imaging modalities. 
A feature of the present invention provides that the 
displaying of the images is performed in real-time, 
so that the image slices change as the surgeon 
moves the anm 34 during surgery. 

Although the arm 34 has been described as 
being usable with the fiducial implants 10A-C, the 
arm 34 also can be used with other existing 
stereotactic localization systems and frames, as 
long as an internal reference point is identifiable. 
As mentioned earlier, an identifiable internal refer- 
ence point is used in order to orient the arm 34 in 
the internal and external coordinate systems. 

FIGS. 5-16 show various flow charts of the 
software used in the system of the present inven- 
tion. The first software flowchart is shown in FIG. 5 
and depicts an interactive, image-guided surgical 
system's main program. The main program begins 
at start 200 and the graphics board is initialized in 
step 201. The mouse is initialized in step 202 and 
the system defaults are set in step 203. The patient 
Information is displayed on a screen In step 204 
and inputs are provided relating to the patient In- 
cluding the patient name, and ID numloer in step 
205. 

In the decision step 206, it is determined 
whether the patient is in the local database. If the 
patient is in the local database, the patient infomna- 
tion and available image sets are displayed in step 
207. At that point, it is detemilned (step 208) 



whether the procedure is the preoperative of in- 
traoperative procedure. If it is an intraoperative 
procedure, the intraoperative software is utilized in 
step 209. 

5 If the patient is not in the local database, it is 

detenmined in decision step 210 whether this is a 
new session or not. If it is a new session, the 
patient information is collected (step 211) and the 
preoperative software is utilized in step 212. If the 

10 session is an old session, archival information Is 
Input (step 213) and the archival infonmation is 
copied into a local database. The program pro- 
ceeds from step 207 in which the patient informa- 
tion and available image sets are displayed. From 

75 the preoperative software 212 and the intraopera- 
tive software 209 a decision step 214 is entered in 
which it Is decided whether or not to process 
another patient. If this session is to be ended, step 
215 is entered to end tiie session. If another patient 

20 is to be processed, the procedure loops back to set 
system defaults step 203. 

The preoperative software of step 212 com- 
prises six major functions. Image slices that depict 
a surgical volume of interest are transferred to the 

25 system hard disk. This involves either directly 
reading ttiem from the storage media or by transfer 
from software. Another function is the displaying of 
the raster images on the display 108. The third 
function is the reviewing of the raster data by 

30 position to set separate threshold and contrast val- 
ues for all raster images. Once the raster data is in 
place, a graphic representation of the raster data is 
generated in the fourtii function. This representa- 
tion can be wire-framed or shaded surface or botii. 

35 The fifth function of the preoperative software is the 
editing of the graphic representation of the raster 
region. Finally, the physician in the sixth function 
may chose to mark regions of interest on tiie raster 
images. These regions are then transferred Into the 

40 graphic image set. The flow charts relating to these 
six functions are shown in FIGS. 6-11. 

FIG. 6 shows tfie overall flow chart of tiie 
preoperative software 212. The first step of the 
preoperative software 212 is the displaying of the 

45 preoperative menu. The I/O source is selected be- 
tween using a mouse or keys In step 21 5. From the 
menu, one of the six functions 216-221 are chosen 
and perfonned. or the exit 222 is selected from the 
menu. From the exit 222 of the menu, it is deter- 
so mined in decision step 223 whether a graphic 
model has been made. If so, there Is a return 224 
to the patient information screen on the display 
108. If no graphic model has been made, it is next 
detemnined in decision step 225 whether actual 

55 scans are on the disk. If tiiey are not, the image set 
is ti^ansfen^ed in step 226. If the actual scans are on 
the disk, a graphic model is made in step 227 and 
tiie patient information screen is returned to. 
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FIG. 7 illustrates the flow chart for the first 
function, the step of adding image sets 216. In step 
228, the Image data is transferred. In decision step 
230, it is determined whether the images are regis- 
tered. If they are not registered, a storage tape is 
loaded (step 232) and if the images are registered, 
a network transfer is performed in step 234. From 
either step 232 or 234, an image header is 
scanned in step 236 to determine the image type, 
the number, the orientation etch The header in- 
fomnation is displayed on display 108 in step 238. 
In step 239. the I/O source is selected between 
mouse or keys. In step 240, the scanned data for 
the new header is extracted. Scan sets are ex- 
tracted in step 241, these scanned sets being 
ordered in step 242. The scanned sets are com- 
pressed to byte width in step 243 and are stored In 
a local database in step 244. In step 245 a decision 
is made whether to operate on another image set 
at which time the first function 216 is either exited 
246 or returned k>ack to step 228. 

FIG. 8 shows the second function in the preop- 
erative software, the display image set function 
217. The first step of this second function is the 
provision of the menu to display images in step 
248. The I/O source is selected in step 249. The 
entire image set can be displayed in step 250 and 
with input 251, the number of images and size of 
the images is determined in step 252. A display 
with a 256 x 256 image is displayed in step 253. 
Altematively, instead of displaying the entire image 
set in step 250, the image set can be displayed in 
a window in step 254. Based upon input in step 
255, the size of the images is detenmined In step 
256 and the display can be a 512 x 512 In a 
requested window in step 257. The second function 
of displaying image sets 217 is exited in step 258. 

FIG. 9 shows a flow chart of the third function, 
the adjust Image display function 218. Step 260 
displays the adjust image menu and step 261 
selects the I/O source. From the menu, the level, 
width, minimum and maximum of the gray scale 
can be adjusted in steps 262-265. /Vfter adjustment, 
gauges are updated in step 266 and the screen is 
updated in step 267 with a return to the menu. Also 
from the menu, there is a reset step 268 that can 
be chosen to restore default values in step 269. 
Rnally, from the menu, there is an exit 270 from 
the adjust image display function 218. 

FIG. 10 illustrates a flow chart for the fourth 
and fifth functions, the making and editing of the 
graphic model. In step 280, an image set is Input. It 
is determined in step 281 whether the image set Is 
axial or not. If it is not. there is a loop until the 
image set is axial. When the image set is axial, the 
gradient difference is used to determine the bound- 
ary of the anatomy of Interest, for example the 
patient's head. In decision step 283, It is deter- 



mined whether to accept vertices for the image 
end. If the vertices are not accepted, a vertex is 
selected and modified In step 284, otherwise, this 
step 284 is skipped to decision step 285 in which 

5 all the images are checked. The image Is then 
incremented to N = N + 1 and the flow loops 
back to the input of decision step 283 if all the 
Images are not checked. Otherwise, when all the 
images are checked as determined in decision step 

10 285. the fourth and fifth functions 219, 220 are 
exited in step 286. 

The sixth function, the marking of the region 
221 is shown as a flow chart in FIG. 11. In 290, a 
target solid is started. Step 291 involves the selec- 

15 tion of a window as an input to the software. The 
process is binded to the selected window in step 
292 and a line is drawn with the mouse in step 
293. The line is encoded in step 294 with all the 
windows having equivalent slices. The input mouse 

20 button status is detemiined in step 295. If the left 
button is selected, there is a loop back to decision 
step 293. If the middle button is selected the 
process loops back to the input step 292. Rnally, if 
the right button is selected, the outline is closed in 

25 step 297 and the target solid is included in step 
298. It is determined in step 299 whether the target 
solid is finished, at which time, the solid is labeled 
in step 200 and the process exited in step 301 . 
The above procedures and flow charts in FIGS. 

30 6-11 describe the preoperative software 212. The 
following flow charts describe the intraoperative 
software 209. The intraoperative software 209 per- 
forms several intraoperative tasks and a true in- 
traprocedural task. The intraoperative tasks include 

35 confirming the patient data and setting up the com- 
puter and surgical device for a specific surgical 
procedure. Tlie software is checked for resets of 
the optical encoders after start-up. This ensures 
accurate angular detennination and surgical tool 

40 end point positioning. The system display setting 
for brightness and contrast are changed to maxi- 
mize image perception within the operating room. 
The graphic display position is rotatable to the 
position which is most intuitive to the surgeon. The 

46 end tool Is also touched to a series of points on a 
cali-bration device in order to determine angular 
errors in each degree of freedom. 

The tasks that are perfomied by the system 
during the operation are the following. After the 

50 patient is positioned, points of commcnality be- 
tween physical space and the image space as 
seen in the display 108 are sequentially located. A 
matrix of rotation between the physical space and 
the image space is calculated. The pointer is then 

55 moved to any position of interest within the surgical 
space. The value of each angular encoder is read 
by the computer 104, the position of the end point 
39 of the surgical tool 38 is calculated in the 
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physical space and then transformed by the matrix 
of rotation to Its equivalent position in the image 
space. This point is shown simultaneously on all 
the raster windows and in the graphic window. 
Additionally, the position of the distal joint Is cal- 
culated and shown on the graphic windows to aid 
the surgeon in orientation. Finally, the surgeon may 
switch to a display of the trajectory of the tool 38. 
This allows the surgeon to detennine an optimal 
path to a point of interest within the surgical or 
image space. 

The flow charts for the intraoperative software 
209 are illustrated in FIGS. 12-16. The overall flow 
chart of the intraoperative software 209 is shown in 
FIG. 12. The first step is the displaying of the 
intraoperative menu in step 310. The I/O source is 
selected in step 311. Five different functions of the 
intraoperative menu can be selected. The first func- 
tion Is the screen setup 312. the second is the ami 
setup 313, the third is the adjust image display 
function 314, the fourth is the graphic model ori- 
entation 315, and the fifth is the use arm function 
316. The intraoperative software is exited in step 
317. 

FIG. 13 shows the first of the intraoperative 
software functions, the screen set up 312. The first 
step 320 is the displaying of the available image 
sets and window assignments. Raster and graphic 
window assignments are input in step 321. From 
here, the information in the windows are updated or 
the screen set of function 312 is exited in step 323. 

FIG. 14 is a flow chart showing the second 
function, the am set up 3*3. In step 325, the anm 
information screen is displayed on display 10B. 
Arm parameters, including the length and type of 
surgical tool, the number of reference points, etc. 
are input in step 326. Information is updated In 
step 327 and a decision is made to exit in decision 
step 328. If it is decided not to exit, the software 
loops back to the input of the update information 
step 327. Otherwise, the second function 313 Is 
exited In step 329. 

The flow chart for the third function, the adjust 
image display function 314, is the same as that of 
FIG. 9 

The flow chart of FIG. 15 shows the fourth 
function, the adjust image display function 315. In 
the first step 330, the menu for this function, the 
orient graphics model menu is displayed. The I/O 
source is selected in step 331. The selection of the 
axis to be rotated is tiien made, with any of the 
X.Y.Z axes being able to be rotated in steps 
332.333,334. The amount of rotation Is input in 
steps 335 and a graphics model orientation is 
updated in step 336. From the display of the menu 
in display 108, the orient graphics model function 
314 is exited in step 337. 

The flow chart of FIG. 16 describes the use 



arm function 316. The first step is ttie displaying of 
the arm information screen 340. The arm hardware 
is initialized in step 341 . The encoders are zeroed 
in step 342. The encoders are moved to tiieir zero 

5 positions and this is input in step 343. In decision 
step 344. it is determined whether all the encoders 
are zeroed, with tiie negative result looping back to 
step 342. If all ttie encoders are zeroed, step 345 Is 
performed to calibrate the encoders. The position 

10 of tile fiducial Implants in physical space, i.e. in tiie 
internal coordinate system, are saved in step 346. 
In decision step 347. it is determined whether all 
ttie fiducial implants 10A-C are located. If they are 
not, the fiduclals are located with the tool 38 and 

15 this information Is Input in step 348. 

When all ttie fiducial implants 10A-C are found, 
the matix of rotation is calculated in step 349, The 
position of the end tip 39 of tiie tool 38 is cal- 
culated In step 350. An input to tiie step 349 (the 

20 calculation of tiie matrix of rotation) is provided in 
step 351 in which the tool 38 is moved. After the 
position of tiie end tip 39 is calculated in step 3?0. 
the position of tiie tool 38 is displayed on the best 
raster image slice and on the graphics model in 

25 step 352. It is then detemnined In decision step 353 
whether tiie trajectory mode is on or not. If it is not, 
it Is detemiined in decision step 354 whether or not 
to exit tiie function 316. If tiie trajectory mode is 
on, a trajectory vector is drawn on the graphics 

30 model in step 355. Once the trajectory vector on 
the graphics model is drawn, it is then determined 
once again whether or not to exit the function 316, 
If it is decided to exit, exit step 356 is tiien ex- 
ecuted. If not, the program loops back to step 349 

35 to calculate a matrix of rotation. 

The above-described flow charts In FIGS. 5-16 
describe an embodiment of software which can be 
used in the present invention. However, other em- 
bodiments of the program to use tiie arm 34 of the 

40 present invention are contemplated. Further, al- 
though the invention 34 has been described as 
being used with fiducial implants 10A-C, the arm 
34 can also be used with any system in which 
intemal points of ence are provided. 

45 

Claims 

1. An interactive system guiding the use of a 
50 surgical tool using at least one imaging technique, 
comprising: 

a mechanical arm having a fixed base at a first end 
and a tool holder that holds the surgical tool at a 
second end; 

55 a display which displays at least one image from 
an image space of a patient's anatomy; and 
a computer coupled to the display and the me- 
chanical arm which tracks the location of tiie sur- 
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glcal tool through physical space, performs a trans- 
forming rotation of the physical space to the image 
space, and causes said display to display the loca- 
tion of the surgical tool within the image space. 

2. The system of claim 1, wherein said mechanical 
arm has joints and means for electrically encoding 
movement of said joints relative to said base, said 
means for encoding being coupled to said com- 
puter to provide said computer with the electrically 
encoded movements.of said joints. 

3. The system of claim 2, wherein the means for 
electrically encoding movement of said joints are 
optical encoders. 

4. The system of claim 3, wherein one optical 
encoder is provided for each degree of freedom of 
said mechanical arm. 

5. The system of claim 1 , wherein said mechanical 
ami is sterilizable. 

6. The system of claim 1, wherein said image from 
an image space is a raster image. 

7. The system of claim 1, wherein a plurality of 
images of the image space are displayed by said 
display with the images being provided by different 
imaging techniques. 

8. The system of claim 7, wherein one of said 
plurality of images is a graphic representation of 
the image space. 

9. A mechanical arm that is manipulable by a 
human operator, with the movement of the arm 
being trackable by a computer coupled to the anm, 
comprising: 

a base that is fixable to a surface: 
a first pair of joints coupled to said base; 
a first arm link coupled at a first end to the first pair 
of joints and having two degrees of freedom pro- 
vided by said first pair of joints; 
a second pair of joints coupled to a second end of 
said first arm linic; 

a second arm linlc coupled at a first end to the 
second pair of joints and having four degrees of 
freedom provided by said first and second pairs of 
joints; 

a third pair of joints coupled to a second end of 
said second arm link; 

a tool holder coupled to said third pair of joints and 
having six degrees of freedom provided by said 
first, second and third pairs of joints; and 
a separate encoder coupled to each joint that elec- 
trically encodes movement of a joint relative to 
another joint and produces signals indicating the 
amount and direction of movement of the joint 

10. The arm of claim 9. wherein said joints are 
gimbal joints. 

11. The am of claim 9. wherein the encoders are 
optical encoders. 

12. The arm of claim 11. wherein the optical en- 
coders are mounted witiiln the gimbal joints. 

13. The arm of claim 12, wherein each optical 



encoder includes a gear tiiat drives the optical 
encoder, and each gimbal joint has a gear that is 
coupled to the gimbal joint and which drives tiie 
optical encoder gear, the gimbal joint gear and the 
5 optical encoder gear meshing at an angle so as to 
prevent backlash. 

14. The arm of claim 13, wherein said angle is 6*. 

15. The arm of claim 13. wherein said gears are 
quarter pitch spur gears. 

70 16. The arm of claim 10, wherein said gimbal joints 
and said first and second arm links are made of 
stainless steel. 

17, The anm of claim 9. wherein said arm is 
sterilizable. 

15 18. The arm of claim 9, wherein said tool holder is 
adapted to hold different types of surgical tools. 

19. A method of performing a surgical procedure 
comprising: 

scanning a portion of a patient's anatomy using an 
20 imaging technique; 

locating an internal point of reference witiiin the 
patient's anatomy using the results of the scanning 

step; 

initializing an end tip of a surgical tool on a ma- 
25 nipulable articulated amn by placing said end tip in 
a known relationship with said internal point of 
reference and noting said initializing in a computer; 
tracking movement in the computer of the end tip 
of the surgical tool through space; 
30 displaying in real time tiie location of tfie end tip of 
the surgical tool witii respect to the scanned por- 
tion. 

20. The mettiod of claim 19. further comprising 
Implanting fiducial implants in the patient's anato- 

35 my to serve as internal points of reference. 

21. The method of claim 20, wherein the tracking 
step includes encoding the movements of joints in 
the arm and sending the encoded movements to 
the computer. 

40 22. The metiiod of claim 21, further comprising 
establishing an internal coordinate system based 
on the fiducial implants. 

23. The method of claim 22, further comprising 
establishing an external coordinate system related 

45 to a fixed base of said arm, the position in space of 
said base being stored upon tiie initializing of said 
end tip. 

24. The metiiod of claim 23, further comprising 
scanning tiie portion of the patient's anatomy using 

50 a plurality of imaging techniques. 

25. The method of claim 24, further comprising 
simultaneously displaying images from each of tiie 
plurality of imaging techniques. 

26. The method of claim 25, furtiier comprising 
55 replacing the surgical tool on said arm with a 

different type of surgical tool. 
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